This invention relates to an image processing apparatus which processes a color image signal, an image processing method, an image processing program thereof and an image pickup apparatus which has a function of processing a color image signal, and more particularly an image processing apparatus, an image processing method, an image processing program and an image pickup apparatus for controlling the adjustment of color signals so as to optimize the color reproduction.
Image pickup apparatus such as a digital still camera and a digital video camera in which a solid-state image pickup element is used frequently have an automatic white balance function. One of such solid-state image pickup apparatus is disclosed in Japanese Patent Laid-Open No. 2002-281512. The automatic white balance function adjusts the color so that a white subject is displayed as a white picture irrespective of the color of the light source upon image pickup.
However, even if a white subject is reproduced as an image of a precise white picture, this does not necessarily mean that an appearance of the white subject to a human being in an image pickup scene is reproduced naturally. For example, a rather reddish white under an incandescent lamp or a rather bluish white in the shade outdoors is sometimes sensed natural. One of reasons of this is that an adaptation mechanism of the visual sense of a human being has a characteristic that it does not always follow up the white in the visual field. Further, also it makes one of the causes that, when a user picks up an image, the intention, liking, and so forth of the user are different such as whether the user wants to have an atmosphere of a scene of a view left or wants to have accurate information of a subject as an article left. Furthermore, not only with regard to the white balance but also with regard to a color other than the white, reproduction of a subject such that it is similar in colorimetry to that under a particular light source does not necessarily result in natural reproduction of the subject in an appearance intended by the user upon image pickup.
In the automatic white balance function, for example, a variation in color by a light source is estimated from an image signal of a result of image pickup, and the estimated color is followed up automatically. In such control as just described, for example, a color temperature range or the like for decision of a light source is frequently set in an image pickup apparatus in advance. If the color temperature range is great, then a resulting picture does not have an atmosphere of the light source left very much therein although it reproduces the color of the subject in fidelity. However, if the color temperature range is small, then a resulting picture does not exhibit a correct color of the subject although it has an atmosphere of the light source left therein.
Meanwhile, an image pickup apparatus which includes a mechanism for intentionally displacing the white balance is conventionally available. Such empirical perception is conventionally known that, for example, under an artificial light source such as an incandescent lamp in a room, where an image is rather reddish, an achromatic color or a color of the skin is reproduced naturally. Thus, such a countermeasure is conventionally available that an offset process for leaving a tinge of red or a process of suppressing the R (Red) gain of the white balance is performed upon follow-up on the low color temperature side. Also another countermeasure is available which allows a user to set a range of movement of the white balance amplifier gain.
In such intentional displacement of the adjustment value by an automatic white balance process as in the techniques described above, efforts have been made to change over the way of displacement of the adjustment value based on fixed circumstantial judgment. However, it is difficult to unconditionally decide a condition which can be regarded appropriate in any situation. Particularly in order to prevent such a burden from being imposed on a user that the user designates a follow-up condition of the white balance, adaptive control to a scene for image pickup is required. To this end, for example, it is a possible countermeasure to classify a color temperature level or a brightness level of a scene based on a relationship in magnitude to a threshold value set in advance and change over the follow-up method depending upon the classification. However, the countermeasure cannot always achieve an optimum state.
It is desirable to provide an image processing apparatus which can automatically produce an image which exhibits a natural white balance state in which the image appears in a color proximate to that which looks to an observer in an image pickup scene.
It is also desirable to provide an image pickup apparatus which can automatically produce an image which exhibits a natural white balance state in which the image appears in a color proximate to that which looks to an observer in an image pickup scene.